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ABSTRACT 



A system allowing a user of a browser program on a 
computer connected to an open distributed hypermedia 
system to access and execute an embedded program object. 
The program object is embedded into a hypermedia docu- 
ment much like data objects. The user may select the 
program object from the screen. Once selected the program 
object executes on the user's (client) computer or may 
execute on a remote server or additional remote computers 
in a distributed processing arrangement. After launching the 
program object, the user is able to interact with the object as 
the invention provides for ongoing interprocess communi- 
cation between the application object (program) and the 
browser program. One application of the embedded program 
object allows a user to view large and complex multi- 
dimensional objects from within the browser's window. The 
user can manipulate a control panel to change the viewpoint 
used to view the image. The invention allows a program to 
execute on a remote server or other computers to calculate 
the viewing transformations and send frame data to the 
client computer thus providing the user of the client com- 
puter with interactive features and allowing the user to have 
access to greater computing power than may be available at 
the user's client computer. 

10 Claims, 9 Drawing Sheets 

Microfiche Appendix Included 
(4 Microfiche, 375 Pages) 
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DISTRIBUTED HYPERMEDIA METHOD objects. In this way, the user is able to navigate easily among 

FOR AUTOMATICALLY INVOKING data objects. The data objects may be local to the user's 

EXTERNAL APPLICATION PROVIDING computer system or remotely located over a network. An 

INTERACTION AND DISPLAY OF early hypertext system is HyperCard, by Apple Computer, 

EMBEDDED OBJECTS WITHIN A 5 Inc. HyperCard is a standalone system where the data objects 

HYPERMEDIA DOCUMENT are local to the user's system. 

When a user selects a phrase in a hypertext document that 

NOTICE REGARDING COPYRIGHTED has ^ associated link to another document, the linked 

MATERIAL document is retrieved and displayed on the user's display 

A portion of the disclosure of this patent document 10 screen. This allows the user to obtain more information in an 

contains material which is subject to copyright protection. efficient and easy manner. This provides the user with a 

The copyright owner has no objection to the facsimile simple, intuitive and powerful way to "branch off' from a 

reproduction by anyone of the patent document or the patent main document to learn more about topics of interest, 

disclosure as it appears in the Patent and Trademark Office Objects may be text, images, sound files, video data, 

file or records, but otherwise reserves all copyright rights 15 documents or other types of information that is presentable 

whatsoever. to a user of a computer system. When a document is 

primarily text and includes links to other data objects 

BACKGROUND OF THE INVENTION according to the hypertext format, the document is said to be 

This invention relates generally to manipulating data in a & hypertext document. When graphics, sound, video or other 

computer network, and specifically to retrieving, presenting 20 media capable of being manipulated and presented in a 

and manipulating embedded program objects in distributed computer system is used as the object linked to, the docu- 

hypermedia systems. ment is said to be a hypermedia document. A hypermedia 

Computer networks are becoming increasingly popular as document is similar to a hypertext document, except that the 

a medium for locating and accessing a wide range of data „, user is able to clickon images, sound icons, video icons, etc., 

from locations all over the world. The most popular global ^at link to other objects of various media types, such as 

network is the Internet with millions of computer systems additional graphics, sound, video, text, or hypermedia or 

connected to it. The Internet has become popular due to hypertext documents. 

widely adopted standard protocols that allow a vast inter- FIG. 1 shows examples of hypertext and hypermedia 

connection of computers and localized computer networks 3Q documents and links associating data objects in the docu- 

to communicate with each other. Computer systems con- ments to other data objects. Hypermedia document 10 

nected to a network such as the Internet may be of varying includes hypertext 20, an image icon at 22, a sound icon at 

types, e.g., mainframes, workstations, personal computers, 24 and more hypertext 26. FIG. 1 shows hypermedia docu- 

etc. The computers are manufactured by different companies ment 10 substantially as it would appear on a user's display 

using proprietary hardware and operating systems and thus 35 screen. The user is able to select, or "click" on icons and text 

have incompatibilities in their instruction sets, busses, on a display screen by using an input device, such as a 

software, file formats and other aspects of their architecture mouse, in a manner well-known in the art. 

and operating systems. Localized computer networks con- When the user clicks on the phrase "hypermedia/' soft- 

nected to the Internet may be incompatible with other ware running on the user's computer obtains the link asso- 

computer systems and localized networks in terms of the ^ ciated with the phrase, symbolically shown by arrow 30, to 

physical layer of communication including the specific access hypermedia document 14. Hypermedia document 14 

hardware used to implement the network. Also, different is retrieved and displayed on the user's display screen. Thus, 

networks use differing, incompatible protocols for transfer- the user is presented with more information on the phrase 

ring information and are not able to communicate with each "hypermedia." The mechanism for specifying and locating a 

other without a translation mechanism such as a "gateway". 45 linked object such as hypermedia document 14 is an HTML 

The Internet provides a uniform and open standard for "element" that includes an object address in the format of a 

allowing various computers and networks to communicate Uniform Resource Locator (URL). 

with each other. For example, the Internet uses Transfer Similarly, additional hypertext 26 can be selected by the 
Control Protocol/Internet Protocol ("TCP/IP") that defines a user to access hypertext document 12 via link 32 as shown 
uniform packet-switched communication standard which is 50 in FIG. 1. If the user selects additional hypertext 26, then the 
ultimately used in every transfer of information that takes text for hypertext document 12 is displayed on the user 
place over the Internet. screen. Note that hypertext document 12, itself, has hyper- 
Other Internet standards are the HyperText Transmission text at 28. Thus, the user can click on the phrase "hyper- 
Protocol ("HTTP") that allows hypertext documents to be media'' while viewing document 12 to access hypermedia 
exchanged freely among any computers connected to the 55 document 14 in a manner similar to that discussed above. 
Internet and HyperText Markup Language ("HTML") that Documents, and other data objects, can be referenced by 
defines the way in which hypertext documents designate many links from many different source documents. FIG. 1 
links to information. See, e.g., Berners-Lee, T. J., "The shows document 14 serving as a target link for both docu- 
world-wide web/' Computer Networks and ISDN Systems ments 10 and 12. A distributed hypertext or hypermedia 
25 (1992). 60 document typically has many links within it that specify 
A hypertext document is a document that allows a user to many different data objects located in computers at different 
view a text document displayed on a display device con- geographical locations connected by a network. Hypermedia 
nected to the user's computer and to access, retrieve and document 10 includes image icon 22 with a link to image 16. 
view other data objects that are linked to hypertext words or One method of viewing images is to include an icon, or other 
phrases in the hypertext document. In a hypertext document, 65 indicator, within the text. 

the user may "click on/' or select, certain words or phrases Typically, the indicator is a very small image and may be 

in the text that specify a link to other documents, or data a scaled down version of the tull image. The indicator may 
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be shown embedded within the text when the text is dis- 
played on the display screen. The user may select the 
indicator to obtain the full image. When the user clicks on 
image icon 22 browser software executing on the user's 
computer system retrieves the corresponding full imag e, 5 
e.g. , a bit map, and displays it by using external software 
called a "v iewer." This results in the full image, represented 
by image 16, being displayed on the screen. 

An example of a browser program is the National Center 
for Supercomputing Application' s (NCSA) Mosaic software 10 
developed by the University of Illinois at Urbana/ 
Champaign, 111. Another example is "Cello" available on the 
Internet at http://www.law.cornell.edu/. Many viewers exist 
that handle various file formats such as ".TIF," ".GIF," 
formats. When a browser program invokes a viewer 15 
program, the viewer is launched as a separate process. The 
view displays the full image in a separate "window" (in a 
windowing environment) or on a separate screen. This 
means that the browser program is no longer active while the 
viewer is active. B y using indicators to act as place holders 20 
fo r follimages thafare retrieved and disji layedja&Lwiien a 
us er selects the indicator, dataJrafEc over the ne twork is 
red uced . Also, since the retrieval and display of large images 
mjiyjeoui re several seconds or more of transfer ti me the user 
does not have to wait to have images transferred that are of 25 
no interest to the user. 

Returning to FIG. 1, another type of data object is a sound 
object shown as sound icon 24 within the hypermedia 
document. When the user selects sound icon 24, the user's 
computer accesses sound data shown symbolically by data 30 
file 40. The accessed sound data plays through a speaker or 
other audio device. 

As discussed above, hypermedia documents allow a user 
to access different data objects. The objects may be text, 35 
images, sound files, video, additional documents, etc. As 
used in this specification, a data object is informati on 
capable of being retrieved*anfi p res ented tn ^ user of a 
co rgpijter sy stem. Some data objects include executable 
code combined with data. An example of such a combination ^ 
is a "self -extracting" data object that includes code to 
"unpack" or decompress data that has been compressed to 
make it smaller before transferring. When a browser 
retrieves an object such as a self -extracting data object the 
browser may allow the user to "launch" the self-extracting 45 
data object to automatically execute the unpacking instruc- 
tions to expand the data object to its original size. Such a 
combination of executable code and data is limited in that 
the user can do no more than invoke the code to perform a 
singular function such as performing the self-extraction after SQ 
which time the object is a standard data object. 

Other existing approaches to embedding interactive pro- 
gram objects in documents include the Object Linking and 
Embedding (OLE) facility in Microsoft Windows, by 
Microsoft Corp., and OpenDoc, by Apple Computer, Inc. At 55 
least one shortcoming of these approaches is that neither is 
capable of allowing a user to access embedded interactive 
program objects in distributed hypermedia documents over 
networks. 

FIG. 2 is an example of a computer network. In FIG. 2, 60 
computer systems are connected to Internet 100, although in 
practice Internet 100 may be replaced by any suitable 
computer network. In FIG. 2, a user 102 operates a small 
computer 104, such as a personal computer or a work 
station. The user's computer is equipped with various 
components, such as user input devices (mouse, trackball, 
keyboard, etc.), a display device (monitor, liquid crystal 
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display (LCD), etc.), local storage (hard disk drive, etc.), and 
other components. Typically, small computer 104 is con- 
nected to a larger computer, such as server A at 106. The 
larger computer may have additional users and computer 
systems connected to it, such as computer 108 operated by 
user 110. Any group of computers may form a localized 
network. A localized network does not necessarily adopt the 
uniform protocols of the larger interconnecting network 
(i.e., Internet 100) and is more geographically constrained 
than the larger network. The localized network may connect 
to the larger network through a "gateway" or "node" imple- 
mented on, for example, a server. 

Internet 100 connects other localized networks, such as 
server a at 120, which interconnects users izz , ii4 ana 126 
and their respective computer systems to Internet 100. 
Internet 100 is made up of many interconnected computer 
systems and communication links. Communication links 
may be by hardwire, fiber optic cable, satellite or other radio 
wave propagation, etc. Data may move from server A to 
server B through any number of intermediate servers and 
communication links or other computers and data processing 
equipment not shown in FIG. 2 but symbolically represented 
by Internet 100. 

A user at a workstation or personal computer need not 
connect to the Internet via a larger computer, such as server 
A or server B. This is shown, for example, by small 
computer 130 connected directly to Internet 100 as by a 
telephone modem or other link. Also, a server need not have 
users connected to it locally, as is shown by server C at 132 
of FIG. 2. Many configurations of large and small computers 
are possible. 

Typically, a computer on the Internet is characterized as 
either a "client" or "server" depending on the role that the 
computer is playing with respect to requesting information 
or providing information. Client computers are computers 
that typically request information from a server computer 
which provides the information. For this reason, servers are 
usually larger and faster machines that have access to many 
data files, programs, etc., in a large storage associated with 
the server. However, the role of a server may also be adopted 
by a smaller machine depending on the transaction. That is, 
user 110 may request information via their computer 108 
from server A. At a later time, server A may make a request 
for information from computer 108. In the first case, where 
computer 108 issues a request for information from server A, 
computer 108 is a "client" making a request of information 
from server A Server A may have the information in a 
storage device that is local to Server A or server A may have 
to make requests of other computer systems to obtain the 
information. User 110 may also request information via their 
computer 108 from a server, such as server B located at a 
remote geographical location on the Internet. However, user 
110 may also request information from a computer, such as 
small computer 124, thus placing small computer 124 in the 
role of a "server." For purposes of this specification, client 
and server computers are categorized in terms of their 
predominant role as either an information requestor or 
provider. Clients are generally information requestors, while 
servers are generally information providers. 

Referring again to FIG. 1, data objects such as distributed 
hypermedia documents 10, 12 and 14, image 16 and sound 
data file 40, may be located at any of the computers shown 
in FIG. 2. Since these data objects may be linked to a 
document located on another computer the Internet allows 
for remote object linking. 

For example, hypertext document 10 of FIG. 1 may be 
located at user 110's client computer 108. When user 110 
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makes a request by, for example, clicking od hypertext 20 
(i.e., the phrase "hypermedia"), user UO's small client 
computer 108 processes links within hypertext document 10 
to retrieve document 14. In this example, we assume that 
document 14 is stored at a remote location on server B. 
Thus, in this example, computer 108 issues a command that 
includes the address of document 14. This command is 
routed through server A and Internet 100 and eventually is 
received by server B. Server B processes the command and 
locates document 14 on its local storage. Server 14 then 
transfers a copy of the document back to client 108 via 
Internet 100 and server A. After client computer 108 
receives document 14, it is displayed so that user U0 may 
view it. 

Similarly, image object 16 and sound data file 40 may 
reside at any of the computers shown in FIG. 2. Assuming 
image object 16 resides on server C when user 110 clicks on 
image icon 22, client computer 108 generates a command to 
retrieve image object 16 to server C. Server C receives the 
command and transfers a copy of image object 16 to client 
computer 108. Alternatively, an object, such as sound data 
file 40, may reside on server A so that it is not necessary to 
traverse long distances via the Internet in order to retrieve 
the data object. 

The Internet is said to provide an "open distributed 
hypermedia system." It is an "open" system since Internet 
100 implements a standard protocol that each of the con- 
necting computer systems, 106, 130, 120, 132 and 134 must 
implement (TCP/IP). It is a "hypermedia" system because it 
is able to handle hypermedia documents as described above 
via standards such as the HTTP and HTML hypertext 
transmission and mark up standards, respectively. Further, it 
is a "distributed" system because data objects that are 
imbedded within a document may be located on many of the 
computer systems connected to the Internet. An example of 
an open distributed hypermedia system is the so-called 
"world-wide web" implemented on the Internet and dis- 
cussed in papers such as the Berners-Lee reference given 
above. 

Hie open distributed hypermedia system provided by the 
Internet allows users to easily access and retrieve different 
data objects located in remote geographic locations on the 
Internet. However, this open distributed hypermedia system 
as it currently exists has shortcomings in that today's large 
data objects are limited largely by bandwidth constraints in 
the various communication links in the Internet and local- 
ized networks, and by the limited processing power, or 
computing constraints, of small computer systems normally 
provided to most users. Large data objects are difficult to 
update at frame rates fast enough (e.g., 30 frames per 
second) to achieve smooth animation. Moreover, the pro- 
cessing power needed to perform the calculations to animate 
such images in real time does not exist on most 
workstations, not to mention personal computers. Today's 
browsers and viewers are not capable of performing the 
computation necessary to generate and render new views of 
these large data objects in real time. 

For example, the Internet's open distributed hypermedia 
system allows users to view still images. These images are 
simple non-interactive two-dimensional images, similar to 
photographs. Much digital data available today exists in the 
form of high-resolution multi-dimensional image data (e.g., 
three dimensional images) which is viewed on a computer 
while allowing the user to perform real time viewing trans- 
formations on the data in order for the user to better 
understand the data. 

An example of such type of data is in medical imaging 
where advanced scanning devices, such as Magnetic Reso- 
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nance Imaging (MRI) and Computed Tomography (CT), are 
widely used in the fields of medicine, quality assurance and 
meteorology to present physicians, technicians and meteo- 
rologists with large amounts of data in an efficient way. 
5 Because visualization of the data is the best way for a user 
to grasp the data's meaning, a variety of visualization 
techniques and real time computer graphics methods have 
been developed. However, these systems are bandwidth- 
intensive and compute-intensive and often require multipro- 

10 cessor arrays and other specialized graphics hardware to 
carry them out in real time. Also, large amounts of secondary 
storage for data are required. The expense of these require- 
ments has limited the ability of researchers to readily 
exchange findings since these larger computers required to 

15 store, present and manipulate images are not available to 
many of the researchers that need to have access to the data. 

On the other hand, small client computers in the form of 
personal computers or workstations such as client computer 
108 of FIG. 2 are generally available to a much larger 

20 number of researchers. Further, it is common for these 
smaller computers to be connected to the Internet. Thus, it 
is desirable to have a system that allows the accessing, 
display and manipulation of large amounts of data, espe- 
cially image data, over the Internet to a small, and relatively 

25 cheap, client computer. 

Due to the relatively low bandwidth of the Internet (as 
compared to today's large data objects) and the relatively 
small amount of processing power available at client 
computers, many valuable tasks performed by computers 

30 cannot be performed by users at client computers on the 
Internet Also, while the present open distributed hyperme- 
dia system on the Internet allows users to locate and retrieve 
data objects it allows users very little, if any, interaction with 
these data objects. Users are limited to traditional hypertext 

35 and hypermedia forms of selecting linked data objects for 
retrieval and launching viewers or other forms of external 
software to have the data objects presented in a comprehen- 
sible way. 

40 Thus, it is desirable to have a system that allows a user at 
a small client computer connected to the Internet to locate, 
retrieve and manipulate data objects when the data objects 
are bandwidth-intensive and compute-intensive. Further, it 
is desirable to allow a user to manipulate data objects in an 

45 interactive way to provide the user with a better understand- 
ing of information presented and to allow the user to 
accomplish a wider variety of tasks. 

SUMMARY OF THE INVENTION 

50 The present invention provides a method for running 
embedded program objects in a computer network environ- 
ment. Hie method includes the steps of providing at least 
one client workstation and one network server coupled to the 
network environment where the network environment is a 
55 distributed hypermedia environment; displaying, on the cli- 
ent workstation, a portion of a hypermedia document 
received over the network from the server, where the hyper- 
media document includes an embedded controllable appli- 
cation; and interactively controlling the embedded control- 
so lable application from the client workstation via 
communication sent over the distributed hypermedia envi- 
ronment. 

The present invention allows a user at a client computer 
connected to a network to locate, retrieve and manipulate 
65 objects in an interactive way. The invention not only allows 
the user to use a hypermedia format to locate and retrieve 
program objects, but also allows the user to interact with an 
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application program located at a remote computer. Interpro- FIG. 10 is a diagram of the various processes and data 

cess communication between the hypermedia browser and paths in the present invention. 

the embedded application program is ongoing after the DETAILED DESCRIPTION OF A PREFERRED 

program object has been launched. The user is able to use a EMBODIMENT 

vast amount of computing power beyond that which is 5 r _ , A ^ t. a * 

A . , . 4t _ , i - A L 375 pages of Source code on 4 microfiche Appendices A 

contained m the user s client computer. ^ fi pjQy]M tQ ^ specificatioa £ am code 

In one application, high resolution three dimensional should be consulted to provide details of a specific embodi- 

images are processed in a distributed manner by several me nt of the invention in conjunction with the discussion of 

computers located remotely from the user's client computer. the routines in this specification. The source code in Appen- 

This amounts to providing parallel distributed processing for 1° tfix a includes NCSA Mosaic version 2.4 source code along 

tasks such as volume rendering or three dimensional image with modifications to the source code to implement the 

transformation and display. Also, the user is able to rotate, present invention. Appendix B includes source code imple- 

scale and otherwise reposition the viewpoint with respect to menting an application program interface. The source code 

these images without exiting the hypermedia browser soft- ^ written in the "C" computer language to run on an 

ware. The control and interaction of viewing the image may 15 X -Window platform. 

be provided within the same window that the browser is F[G 3 is m illustration of a computer system suitable for 
using assuming the environment is a "windowing" environ- use with me present invention. FIG. 3 depicts but one 
ment. The viewing transformation and volume rendering example of many possible computer types or configurations 
calculations may be performed by remote distributed com- cap able of being used with the present invention. FIG. 3 
puter systems. 20 SD0WS computer system 150 including display device 153, 
Once an image r epresenting a new viewpoint is computed display screen 155, cabinet 157, keyboard 159 and mouse 
the frame image is'transmitted over the networkJothejisei^s 161. Mouse 161 and keyboard 159 are "user input devices/' 
clien t computer where it is displaye d at a designated position Other examples of user input devices are a touch screen, 
within a hypermedia document. By transmitting only light pen, track ball, data glove, etc. 
^eno ugh information to update the image , tne need tor a nign Mouse 161 may have one or more buttons such as buttons 
^/bandwidth data connection is reduced. Compression can be 163 shown in FIG. 3. Cabinet 157 houses familiar computer 
- used to further reduce the bandwidth requirements for data components such as disk drives, a processor, storage means, 
fi/ transmission. etc. As used in this specification "storage means" includes 
Other applications of the invention are possible. For 3Q any storage device used in connection with a computer 
example, the user can operate a spreadsheet program that is system such as disk drives, magnetic tape, solid state 
being executed by one or more other computer systems memory, bubble memory, etc. Cabinet 157 may include 
connected via the network to the user's client computer. additional hardware such as input/output (I/O) interface 
Once the spreadsheet program has calculated results, the cards for connecting computer system 150 to external 
results may be sent over the network to the user's client 35 devices such as an optical character reader, external storage 
computer for display to the user. In this way, computer devices, other computers or additional devices, 
systems located remotely on the network can be used to FIG. 4 is an illustration of basic subsystems in computer 
provide the computing power that may be required for system 150 of FIG. 3. In FIG. 4, subsystems are represented 
certain tasks and to reduce the data bandwidth by only by blocks such as central processor 180, system memory 181 
transmitting results of the computations. ^ consisting of random access memory (RAM) and/or read- 
Still other applications of the present invention are only memory (ROM), display adapter 182, monitor 183 
possible, as disclosed in the specification, below. (equivalent to display device 153 of FIG. 3), etc. The 

subsystems are interconnected via a system bus 184. Addi- 

BRIEF DESCRIPTION OF THE DRAWINGS tional subsystems such as a printer, keyboard, fixed disk and 

T71-/-1 .« . . ! r U 4 . J. . -Ac others are shown. Peripherals and input/output (I/O) devices 

FIG. 1 illustrates examples 01 hypertext and hypermedia 4i . t , 4 , *_ . . \? 1 

, , j 1 * 1 can be connected to the computer system by, for example 

documents and finks; . , _ 1 . 1 * i« i_ /* 

serial port 185. For example, serial port 185 can be used to 

FIG. 2 is an example of a computer network; connect the computer system to a modem for connection to 

FIG. 3 is an illustration of a computer system suitable for a network or serial p ort 185 can be used to i nterf ace with a 

use with the present invention; 5Q mouse input device, Ine interconnection via system bus 184 

FIG. 4 is an illustration of basic subsystems in the allows central processor 180 to communicate with each 

computer system of FIG. 3; subsystem and to control the execution of instructions from 

FIG. 5 is an illustration of an embodiment of the invention svstem memory 181 or fixed disk 186, and the exchange of 

using a client computer, server computer and a network; information between subsystems. Other arrangements of 

c , , , . . c ,u * • <r< subsystems and interconnections are possible 

FIG. 6 shows another embodiment of the present inven- 55 J « • n * a r 



tion using additional computers on the network; 



FIG. 5 is an illustration of an embodiment of the invention 

_ . _ , , _ , . . using a client computer, server computer and a network. 

FIG. 7 A is a flowchart of some of the functionality within 1 Wi c v * * mn • * *u 

the HTML arse c file- client computer 200 communicates with server 

„ . , ' . » . , ... computer 204 via network 206. Both clie nt compute r 200 

u ^Z?f a fl0W ^! art 0f 501X10 0f thc fanctlOMht y 60 and server computer 204 use 'a networr^oto^llayer to 

the HTMLformatc file; communicate with network 206. l n__a preferred emhnrWnt _ 

FIG. 8 A is a flowchart of some of the functionality within n etwork 206 is the Internet and thTnetwork protocol lay ers 

the HTMLwidget.c file; are TCP/IP. Other networks and network protocols may be 

FIG. 8B is a flowchart of some of the functionality within used. For ease of illustration, additional hardware and soft- 

the HTML.c file; 65 ware layers are not shown in FIG. 5. 

FIG. 9 is a screen display generated in accordance with Client computer 200 includes processes, such as browser 

the present invention; and client 208 and application client 210. In a preferred 
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embodiment, application client 210 is resident within client to display the multidimensional embryo data on the display 

computer 200 prior to browser client 208's parsing of a screen to a user of the client computer 200. The user is then 

hypermedia document as discussed below. In a preferred able to interactively operate controls to recompute different 

embodiment application client 210 resides on the hard disk views for the image data. In a preferred embodiment, a 

or RAM of client computer 200 and is loaded (if necessary) 5 control window is displayed within, or adjacent to, a win- 

and executed when browser client 208 detects a link to dow generated by browser client 208 that contains a display 

application client 210. The preferred embodiment uses the of hypermedia document 212. An example of such display 

XEvent interprocess communication protocol to exchange I s <^ssed below in connection with FIG. 9. Thus, the user 

information between browser client 208 and application ^ able to interacbvely mampulate a mulUdimensional image 

client 210 as described in more detail, below. Another 10 object by meaiis of the present invention. In order to make 

possibility is to install application client 210 as a "terminate WP^hon client 210 integral with displays created by 

and stay resident" (TSR) program in an operating system browser ckent 208 > both ^ browser chent md ^ ^ 

environment, such as X-Window. Thereby making access to c * tlon <*ent mu5t be m communication with each other, as 

application client 210 much faster. showa b ? the arrow connecting the two within chent com- 

„ t „ 0 . , * ... « P uter 200. The manne r of communication^slhcough an 

Browser chent 208 is a process that a user of chent 15 aDpl ication BcSSTinterfta T/APrt. disci^dhe^ T 

computer 200 invokes in order to access various data .iLr — \. . — i— « — , 7™7T\ . 

u* ♦ i u a a * *m\£ Browser client 208 is a process, such as NCSA Mosaic, 

objects, such as hypermedia documents, on network 206. „ n A . . . . * , . ? 9 

Hypermedia document 212 shown within client computer CeUo ' « te - A » 1,c f ™ ^ 21 ° f /^£f d l " s S* wa 5 

200 is an example of a hypermedia document, or object, that ?Kse , n ^ u ™der development called "VIS" and Tanel" 

a user has requested access to. In this example, hypermedia 20 ^ f Center for Knowledge Management at the 

document 212 has been retrieved from a server connected to ^^T*' S °*?. can ™ c °> as P a *. of lhe y ' 

network 206 and has been loaded into, e.g., client computer G™up s djstributed hypermedia object embedding approach 

iftft-.DAu^M^^^ ^,nn 0 described m "Integrated Control of Distributed Volume 

200 sRAM or other storage device. Visualization!^ 

Once hypermedia document 212 has been loaded into _ Martin> Mt Doylc; t0 ^ published in the Proceedings of 

chent computer 200, browser chent 208 parses hypermedia visualization 1994, IEEE Press, Washington, D.C., October 

document 212. In parsing hypermedia document 212, 1994 

browser client 208 detects links to data objects as discussed \/™\™ „„,i j_ • f - t . „r , , . 

, - iL n 1 j r. L t a. " r * Versions and descriptions of software embodying the 

above in the Background of the Invention section. In FIG. 5, m . . n 1 , j . A 

, j j * • 1 1 .11, present invention are generally available as hyperlinked data 1 

hypemiedia document 212 mcludes an embedded program objec|s from ^ Visib]e ^ ^ w;dc Wcb | 

hnk a 214. Embedded program link 214 identifies apphca- document at the VRL J dress « H TTP://visembryo. 

tion client 212 as an application to invoke. In this present ucg f g^y** 

example, the application, namely, application client 210, . " " 4 , . 

resides on the same computer as the browser client 208 that emb ° dunent <* the present invention uses an 

the user is executing to view the hypermedia document. fPP hca f? n server process executing on server computer 204 

Embedded program link 214 may include additional 35 tojssus processmg mat may need to be performed by an 

information, such as parameters, that tell application client external program. For example, m FIG. 5 application server 

210 how to proceed. For example, embedded program link 220 / esldes ° n server computer 204. Application server 220 

214 may include a specification as to a data object that wori ? " «*™»"«^ with application chent 210 residing 

application client 210 is to retrieve and process. oa , cheni com %T- 20 °, 5 7 | r 0 eferred "bodiment, apph- 

. ,. ... , , , , 40 cation server 220 is called VRServer, also a part of Doyle 

When browser chent 208 encounters embedded program G , & h since (er ^ ^ a 

link 214, it invokes apphcation client 210 (optionally, with larger computer having more data processing capabilities 

parameters or other information) and application chent 210 ^ j £ cj lication gBpm 22Q can 

executes instructions to perform processing in accordance operate mMe efficientl ^ mucn faster> man application 

with the present invention. 4J ^ 21Q ^ cxecuting and numerous ^tnic 

An example of the type of processing thai annlicati nn fans. 
chent 210 may perform is multidimensional imaeeTisual- ln me nt , le where a multidimensional i^ge 
izaGon. iNote mat appneauon client 210 is in communication objecl representing medical data for an embryo fc being 
with network 206 via the network protocol layer of client viewedj a pp lication 2 20 could perform much of the 
computer 200. This means that application chent 210 can 50 viewine transformation and volume rendering calculations 
make requests over network 206 for data objects, such as t0 ^ a user to interactively view the embryo data at their 
mulndunensional tmage objects. For example, application client computer display In a pre f erre d embodiment, 
chent 210 may request an object, such as object 1 at 216, applicauon client 2 10 receives signals from a user input 
located in server computer 204. Application client 210 may device at the wa>s client 20 0. An example of such 
make the request by any suitable means. Assuming network ss mpm would be to rota(e me emb ^ ftom a 
206 is the Internet, such a request would typically be made position t0 ane w position from the user's point of view. This 
by using HTTP in response to a HTML-style link definition information is received by apphcation client 210 and pre- 
fer embedded program link 214. ^ generate a ^ over nelwork 20 6 to 

Assuming application client 210 has made a request for application server 220. Once application server 220 receives 

me data object at 216, server process 218 ultimately receives 6 o the information in the form of, e.g., a coordinate transfor- 

the request Server process 218 then retrieves data object mation for a new viewing position, apphcation server 220 

216 and transfers it over network 206 back to application performs the mathematical calculations to compute a new 

^ chent 210. T o continue with the example of a multidime n- view for the embryo image. Once the new view has been ' 

sional visualization appneauon, data object 21^ may be a computed, the image data for the new view is sent over 

*^three dimensional vie w of medical data for, e.g., an embryo . 65 network 206 to application client 210 so that application 

After application client 210 receives the multidimensional client 210 can update the viewing window currently dis- 

data object 216, application client 210 executes instructions playing the embryo image. In a preferred embodiment, 



5^ 
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application server 220 computes a frame buffer of raster together with an associated image, within a hypermedia 

display data, e.g., pixel values, and transf ers this fram e document, allowing objects within the MetaMAP-processed 

buffer to application client 210 . Techniques, such as data image to have URL addresses associated with them. When 

compression ana aeita encoding, can be used to compress a user clicks with a mouse upon an object within the 

the data before transmitting over network 206 to reduce the 5 MetaMAP-processed image, the MetaMAP client applica- 

bandwidth requirement. uon re i ays mc relevant URLback to the hypermedia browser 

It will be readily seen that application server 220 can application, which then retrieves the HTML file or hyper- 

advantageously use server computer 204*s computing media OD j ect wn i cb corresponds to that URL. 

resources to perform the viewing transformation much more ™, . . r . . . . 

. ., t t r A . r. ^ 1- x The vanous processes m the system of the present inven- 

qmcldythanrculdappkcaUo^^ communicate through a custom API called Mosaic/ 

computer^OO^rth^^^ Application Program Interface MEAPI. The 

fra™ buffer coptomng a-^ ^ of defined m s ^ shown ^ 

meTamount of data sent over network:20oLis reduced . By „ - . T 

. — / ._~.-t-— *~~ 4 , . , J Table I. 
using appropriate compression techniques, such as, e.g., 

MPEG (Motion Picture Experts Group) or JPEG (Joint 15 TABLE I 
Photographic Experts Group), efficient use of network 206 is 

preserved. Message Function Message Name 

FIG. 6 shows yet another embodiment of the present 

inventioa FIG. 6 is similar to FIG. 5, except that additional Messages from s & rv« to client: 

computers 222 and 224 are illustrated. Each additional 20 1. Server Update Done XtNrefreshNotify 

Computer includes a process labeled "Application 2. Server Ready XtNpanetStartNotify 

(Distributed)." The distributed application performs a por- 3 - Seiver xtNpanetExitNotify 

lion of the task that an application, such as application server Messages from client to server 

220 or application client 210, perform. In the present. 4> stowQ xtNmapNotify 

example, tasks such as volume rendering may be broken up 25 5 - Ar** Hidden xtNunmapNotify 

and easily performed among two or more computers. These 6 - Area Destroyed XtNexitNotify 

computers can be remote from each other on network 206. . 

Thus, several computers, such as server computer 204 and The messages in Table I are defined in the file protocol_Jib.ta 

additional computers 222 and 224 can all work together to i n Appendix B. The functions of the MEAPI are provided in 

perform the task of computing a new viewpoint and frame 30 protocoLJib.c of Appendix B. Thus, by using MEAPI a 

buffer for the embryo for the new orientation of the embryo server process communicates to a client application program 

image in the present example. The coordination of the to ] et me cuent application know when the server has 

distributed processing can be performed at client computer finished updating information, such as an image frame 

200 by application client 210, at server computer 204 by buffer, or pixmap (Message 1); when the server is ready to 

application server 220, or by any of the distributed appli- 35 s tart processing messages (Message 2) and when the server 

cations executing on additional computers, such as 222 and # exiting or stopping computation related to the server 

224. In a preferred embodiment, distributed processing is application program. 

coordinated by a program called "VIS" represented by For client to server communications, MEAPI provides for 

application client 210 in FIG. 6. the client informing the server when the image display 

Other applications of the invention are possible. For 40 window area is visible, when the area is hidden and when the 

example, the user can operate a spreadsheet program that is area is destroyed. Such information allows the server to 

being executed by one or more other computer systems decide whether to allocate computing resources for, e.g., 

connected via the network to the user's client computer. rendering and viewing transformation tasks where the server 

Once the spreadsheet program has calculated results, those is running an application program to generate new views of 

results may be sent over the network to the user's client 45 a multi dimensional object. Source code for MEAPI funda- 

computer for display within the hypermedia document on mental functions such as handle_client_jnsg, register^ 

the user's client computer. In this way, computer systems client, register_clienl_msg_callback and send_client_ 

located remotely on the network can be used to provide the msg may be found in protocoLJib.c as part of the source 

computing power that may be required for certain tasks and code in Appendix B. 

to reduce the data bandwidth required by only transmitting 50 Next, a discussion of the software processes that perform 

results of the computations. parsing of a hypermedia document and launching of an 

An other type of possible application of this invention application program is provided in connection with Table II 

wo uld involve emoeaaing a program which runs only on th e and FIGS. 7A, 7B, 8A and 8B. 

clie nt machine, but which provides the user with more Table II, below, shows an example of an HTML tag 

functionality than exists in the hypermedia browser alone. 55 format used by the present invention to embed a link to an 

An example of this is an embedded client application which application program within a hypermedia document, 
is capable of viewing and interacting with images which 

have been processed with Dr. Doyle's MetaMAP invention TABLE II 
(U.S. Pat. No. 4,847,604). This MetaMAP process uses 

object-oriented color map processing to allow individual 60 
color index ranges within paletted images to have object 
identities, and is useful for the creation of, for example, 
interactive picture atlases. It is a more efficient means for 
defining irregular "hotspots" on images than the ISMAP 

function of the World Wide Web, which uses polygonal 65 

outlines to define objects in images. A MetaMAP-capable As shown in Table II, the EMBED tag includes TYPE, 

client-based image browser application can be embedded, HREF, WIDTH and HEIGHT elements. The TYPE element 



<EMBED 




TYPE > 


-type" 
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"href 
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« width 


HEIGHT 


- height 


> 
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is a Multipurpose Internet Mail Extensions (MIME) type. scope of the invention. Additional considerations in imple- 

Examples of values for the TYPE element are "application/ menting the method described by the flowchart in software 

x-vis" or "video/mpeg". The type "application /x-vis" indi- may dictate changes in the selection and order of steps, 

cates that an application named "x-vis" is to be used to Some considerations are event handling by interrupt driven, 

handle the object at the URL specified by the HREF. Other 5 polled, or other schemes. A multiprocessing or multitasking 

types are possible such as "application/x-inventor", environment could allow steps to be executed "concur- 

"apphcatiorVpostscript'' etc. In the c ase where TYPE is rentlv " For ease of discussion the implementation of each 

"apphcation/x-vis" this mcMsTfiSTtETohaSct at the URL flowchart may be referred to as if implemented in a single 

address is a three dimensional image object since the pro- routine 

•^-n — ■*"tt* ! XT" — ■ — r — r — t — t~z — 7 T7 * The modifications to NCSA Mosaic version 2.4 software 

gra^x-vis is a daU visualization^ 10 flleg HmL HTMLformat.c, HTMLwidget.c and 

opiate aimensionaUmage^bjects. However, any manner HTML.C will next be discussed, in turn, 

ol application program may De specified by the TYPE Retumm g to FIG. 7, it is assumed that a hypermedia 

element so that other types of applications such as a document has been obtained at a user's client computer and 

spreadsheet program, database program, word processor, mat a browser program executing on the client computer 

etc. may be used with the present invention. Accordingly, the 15 displays the document and calls a first routine in the HTM- 

object reference by the HREF element would be, Lparse.c file called "HTMLparse". This first routine, 

respectively, ;a spreadsheet object, database object, word HTMLparse, is entered at step 252 where a pointer to the 

processor document object, etc. start of the document portion is passed. Steps 254, 256 and 

On the other hand , TYP E values such as "video/mpe g", 258 represent a loop where the document is parsed or 

" image/ gir J , "video/x-sgi-movie'', etc. describe the typejj f 20 scanned for HTML tags or other symbols. While the file 

data that HREF specifies . This is useful where an external HTMLparse.c includes routines to handle all possible tags 

application program, such as a video player, needs to know and symbols that may be encountered, FIG. 7A, for 

what format the data is in, or where the browser client needs simplicity, only illustrates the handling of EMBED tags, 

to determine which application to launch based on the data Assuming there is more text to parse, execution proceeds 

format. Thus, the TYPE value can specify either an app li- 25 to step 256 where routines in HTMLparse.c obtain the next 

cation program or a data type . Other TYPE values aie item (e.g., word, tag or symbol) from the document. At step 

possible. HREF specifies a URL address as discussed above 258 a check is made as to whether the current tag is the 

for a data object. Where TYPE is "application/x-vis" the EMBED tag. If not, execution returns to step 254 where the 

URL address specifies a multi-dimensional image object. next tag in the document is obtained. If, at step 258, it is 

Where TYPE is "video/mpeg" the URL address specifies a 30 determined that the tag is the EMBED tag, execution pro- 

video object. ceeds to step 260 where an enumerated type is assigned for 

WIDTH and HEIGHT elements specify the width and the tag. Each occurrence of a valid EMBED tag specifies an 

height dimensions, respectively, of a-Distributed Hyperme- embedded object. HTMLParse calls a routine "get__mark" 

dia Object Embedding (DHOE) window to display an exter- in HTMLparse.c to put sections of HTML document text 

nal application object such as the three dimensional image 35 into a "markup" text data structure. Routine get_mark, in 

object or video object discussed above. turn, calls ParseMarkType to assign an enumerated type. The 

FIG. 7A is a flowchart describing some of the function- enumerated type is an identifier with a unique integer 

ality within the HTMLparse.c file of routines. The routines associated with it that is used in Later processing described 

in HTMLparse.c perform the task of parsing a hypermedia below. 

document and detecting the EMBED tag. In a preferred 40 Once all of the hypermedia text in the text portion to be 

embodiment, the enhancements to include the EMBED tag displayed has been parsed, execution of HTMLparse.c rou- 

are made to an HTML library included in public domain tines terminates at step 262. 

NCSA Mosaic version 2.4. Note that much of the source FIG. 7B is a flowchart of routines in file HTMIiormat.c 

code in is pre-existing NCSA Mosaic code. Only those to process the enumerated type created for the EMBED tag 

portions of the source code that relate to the new function- 45 by routines in HTMLparse.c. In the X- Window implemen- 

ality discussed in this specification should be considered as tation of a preferred embodiment, the enumerated type is 

part of the invention. The new functionality is identifiable as processed as if it is a regular Motif/XT widget. For details 

being set off from the main body of source code by condi- on X-Window development see, e.g., "Xlib Programming 

tional compilation macros such as "#ifdef . . . #endif" as will Manual," "X Toolkit Intrinsics Programming Manual*' and 

be readily apparent to one of skill in the art. 50 "Motif Programming Manual" published by O'Reilly & 

In general, the flowcharts in this specification illustrate Associates, Inc. HTMLformat is entered at step 270 where 

one or more software routines executing in a computer a pointer to the enumerated type to process is passed, 

system such as computer system 1 of FIG. 1. The routines At step 272 the parameters of the structure are initialized 

may be implemented by any means as is known in the art. in preparation for inserting a DrawingArea widget on an 

For example, any number of computer programming 55 HTML page. This includes providing values for the width 

languages, such as "C", Pascal, FORTRAN, assembly and height of a window on the display to contain an image, 

language, etc., may be used. Further, various programming position of the window, style, URL of the image object, etc. 

approaches such as procedural, object oriented or artificial Various codes are also added by routines in HTMLformat.c 

intelligence techniques may be employed. (such as TriggerMarkChanges) to insert an internal repre- 

The steps of the flowcharts may be implemented by one 60 sentation of the HTML statement into an object list main- 

or more software routines, processes, subroutines, modules, tained internally by the browser. In the X-Window applica- 

etc. It will be apparent that each flowchart is illustrative of tion corresponding to the source code of Appendix A, the 

merely the broad logical flow of the method of the present browser is NCSA Mosaic version 2.4. 

invention and that steps may be added to, or taken away FIG. 8A is a flowchart foT routine HTMLwidget. HTM- 

from, the flowcharts without departing from the scope of the 65 Lwidget creates display data structures and launches an 

invention. Further, the order of execution of steps in the external application program to handle the data object 

flowcharts may be changed without departing from the specified by the URL in the EMBED tag. 
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HTMLwidget is entered at step 280 after HTMLformat 
has created the internal object representation of the EMBED 
tag. HTMLwidget is passed the internal object and performs 
its processing on the object. At step 282 the DrawingArea 
widget is created according to the type of the internal 5 
representation, from HTMLformat, specified in the internal 
object. Similarly, at step 284 a pixmap area for backing 
storage is defined. 

At step 286 a check is made as to whether the type 
attribute of the object, i.e., the value for the TYPE element 10 
of the EMBED tag, is an application. If so, step 290 is 
executed to launch a predetermined application. In a pre- 
ferred embodiment an application is launched according to 
a user-defined list of application type/application pairs. The 
list is defined as a user-configurable XResource as described 15 
in "Xlib Programming Manual." An alternative embodiment 
could use the MIME database as the source of the list of 

application type/application pairs. The routine "vis starL_ 

external_appHcation" in file HTMLformat.c is invoked to 
match the application type and to identify the application to 
launch. 

The external application is started as a child process of the 
current running process (Mosaic), and informed about the 
window ID of the DrawingArea created in HTMLformat. 
The external application is also passed information about the 
ID of the pixmap, the data URL and dimensions. Codes for 
communication such as popping-up/iconifying, start 
notification, quit notification and refresh notification with 
external applications and DrawingArea refreshing are also 
added. Examples of such codes are (1) "setup/start" in 
vis_register_client and vis__get panel_window in HTM- 
Lwidgets.c; (2) "handle messages from external applica- 
tions" in vis_handle panel_msg in HTMLwidgets.c; (3) 
"send messages to external applications" in vis_send_jnsg 
in HTMLwidgets.c; (4) "teiminate external applications" in 
vis_exit in HTMLwidgets.c which calls vis_send_msg to 
send a quit message; and (5) "respond to refresh msgs" in 
vis_redraw in HTMLwidgets.c. 

If, at step 286, the type is determined not to be an 
application object (e.g., a three dimensional image object in 
the case of application "x-vis") a check is made at step 288 
to determine if the type is a video object. If so, step 292 is 
executed to launch a video player application. Parameters 
are passed to the video player application to allow the player 
to display the video object within the DrawingArea within 
the display of the portion of hypermedia document on the 
client's computer. Note that many other application objects 
types are possible as described above. 

FIG. 8B is a flowchart for routine HTML. Routine HTML 
takes care of "shutting down" the objects, data areas, etc. 
that were set up to launch the external application and 
display the data object. HTML is entered at step 300 and is 
called when the display or other processing of the EMBED 
tag has been completed. At step 302 the display window is 
removed and the memory areas for the pixmap and internal 
object structure is made free for other uses. Completion of 
processing can be by user command or by computer control. 

The present invention allows a user to have interactive 
control over application objects such as three dimensional 
image objects and video objects. In a preferred embodiment, 
controls are provided on the external applications' user 
interface. In the case of a VIS/panel application, a process, 
"panel" creates a graphical user interface (GUI) thru which 
the user interacts with the data. The application program, 
VIS, can be executing locally with the user's computer or 
remotely on a server, or on one or more different computers, 
on the network. The application program updates pixmap 
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data and transfers the pixmap data (frame image data) to a 
buffer to which the browser has access. The browser only 
needs to respond to the refresh request to copy the contents 
from the updated pixmap to the DrawingArea. The Panel 
process sends messages as "Msg" sending performed by 
routines such as vis_send_msg and vis__handle panel_msg 
to send events (mousemove, keypress, etc.) to the external 
application. 

FIG. 9 is a screen display of the invention showing an 
interactive application object (in this case a three dimen- 
sional image object) in a window within a browser window. 
In FIG. 9, the browser is NCSA Mosaic version 2.4. The 
processes VIS, Panel and VRServer work as discussed 
above. FIG, 9 shows screen display 356 Mosaic window 350 
containing image window 352 and a portion of a panel 
window 354. Note that image window 352 is within Mosaic 
window 350 while panel window 354 is external to Mosaic 
window 350. Another possibility is to have panel window 
354 within Mosaic window 350. By using the controls in 
panel window 354 the user is able to manipulate the image 
within image window 352 in real time do perform such 
operations as scaling, rotation, translation, color map 
selection, etc. In FIG. 9, two Mosaic windows are being used 
to show two different views of an embryo image. One of the 
views is rotated by six degrees from the other view so that 
a stereoscopic effect can be achieved when viewing the 
images. Communication between Panel and VIS is via 
"Tooltalk" described in, e.g., "Tooltalk 1.1.1 Reference 
Manual," from SunSoft. 

FIG. 10 is an illustration of the processes VIS, Panel and 
VRServer discussed above. As shown in FIG. 10, the 
browser process, Mosaic, communicates with the Panel 
process via inter-client communication mechanisms such as 
provided in the X-Window environment. The Panel process 
communicates with the VIS process through a communica- 
tions protocol (ToolTalk, in the preferred embodiment) to 
exchange visualization command messages and image data. 
The image data is computed by one or more copies of a 
process called VRServer that may be executing on remote 
computers on the network. VRServer processes respond to 
requests such as rendering requests to generate image seg- 
ments. The image segments are sent to VIS and combined 
into a pixmap, or frame image, by VIS. The frame image is 
then transferred to the Mosaic screen via communications 
between VIS, Panel and Mosaic. A further description of the 
data transfer may be found in the paper ''Integrated Control 
of Distributed Volume Visualization Through the World- 
Wide-Web," referenced above. 

In the foregoing specification, the invention has been 
described with reference to a specific exemplary embodi- 
ment thereof. It will, however, be evident that various 
modifications and changes may be made thereunto without 
departing from the broader spirit and scope of the invention 
as set forth in the appended claims. For example, various 
programming languages and techniques can be used to 
implement the disclosed invention. Also, the specific logic 
presented to accomplish tasks within the present invention 
may be modified without departing from the scope of the 
invention. Many such changes or modifications will be 
readily apparent to one of ordinary skill in the art. The 
specification and drawings are, accordingly, to be regarded 
in an illustrative rather than a restrictive sense, the invention 
being limited only by the provided claims. 

What is claimed is: 

1. A method for running an application program in a 
computer network environment, comprising: 

providing at least one client workstation and one network 
server coupled to said network environment, wherein 
said network environment is a distributed hypermedia 
environment; 
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executing, at said client workstation, a browser 
application, that parses a first distributed hypermedia 
document to identify text formats included in said 
distributed hypermedia document and for responding to 
predetermined text formats to initiate processing speci- 5 
fied by said text formats; utilizing said browser to 
display, on said client workstation, at least a portion of 
a first hypermedia document received over said net- 
work from said server, wherein the portion of said first 
hypermedia document is displayed within a first 
browser-controlled window on said client workstation, 
wherein said first distributed hypermedia document 
includes an embed text format, located at a first location 
in said first distributed hypermedia document, that 
specifies the location of at least a portion of an object 
external to the first distributed hypermedia document, 
wherein said object has type information associated 
with it utilized by said browser to identify and locate an 
executable application external to the first distributed 
hypermedia document, and wherein said embed text 2Q 
format is parsed by said browser to automatically 
invoke said executable application to execute on said 
client workstation in order to display said object and 
enable interactive processing of said object within a 
display area created at said first location within the 
portion of said first distributed hypermedia document 
being displayed in said first browser-controlled win- 
dow. 

2. The method of claim 1, wherein said executable appli- 
cation is a controllable application and further comprising 
the step of: 

interactively controlling said controllable application on 
said client workstation via inter-process communica- 
tions between said browser and said controllable appli- 
cation. 

3. The method of claim 2, wherein the communications to 
interactively control said controllable application continue 
to be exchanged between the controllable application and 
the browser even after the controllable application program 
has been launched. ^ 

4. The method of claim 3, wherein additional instructions 
for controlling said controllable application reside on said 
network server, wherein said step of interactively controlling 
said controllable application includes the following sub- 
steps: 45 

issuing, from the client workstation, one or more com- 
mands to the network server; 

executing, on the network server, one or more instructions 
in response to said commands; 

sending information from said network server to said 50 
client workstation in response to said executed instruc- 
tions; and processing said information at the client 
workstation to interactively control said controllable 
application. 

5. The method of claim 4, wherein said additional instruc- 55 
tions for controlling said controllable application reside on 
said client workstation. 

6. A computer program product for use in a system having 
at least one client workstation and one network server 
coupled to said network environment, wherein said network 60 
environment is a distributed hypermedia environment, the 
computer program product comprising: 

a computer usable medium having computer readable 
program code physically embodied therein, said com- 
puter program product further comprising: 65 
computer readable program code for causing said client 
workstation to execute a browser application to parse 
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a first distributed hypermedia document to identify 
text formats included in said distributed hypermedia 
document and to respond to predetermined text for- 
mats to initiate processes specified by said text 
formats; 

computer readable program code for causing said client 
workstation to utilize said browser to display, on said 
client workstation, at least a portion of a first hyper- 
media document received over said network from 
said server, wherein the portion of said first hyper- 
media document is displayed within a first browser- 
controlled window on said client workstation, 
wherein said first distributed hypermedia document 
includes an embed text format, located at a first 
location in said first distributed hypermedia 
document, that specifies the location of at least a 
portion of an object external to the first distributed 
hypermedia document, wherein said object has type 
information associated with it utilized by said 
browser to identify and locate an executable appli- 
cation external to the first distributed hypermedia 
document, and wherein said embed text format is 
parsed by said browser to automatically invoke said 
executable application to execute on said client 
workstation in order to display said object and enable 
interactive processing of said object within a display 
area created at said first location within the portion of 
said first distributed hypermedia document being 
displayed in said first browser-controlled window. 

7. The computer program product of claim 6, wherein said 
executable application is a controllable application and 
further comprising: 

computer readable program code for causing said client 
workstation to interactively control said controllable 
application on said client workstation via inter-process 
communications between said browser and said con- 
trollable application. 

8. The computer program product of claim 7, wherein the 
communications to interactively control said controllable 
application continue to be exchanged between the control- 
lable application and the browser even after the controllable 
application program has been launched. 

9. The computer program product of claim 8, wherein 
additional instructions for controlling said controllable 
application reside on said network server, wherein said step 
of interactively controlling said controllable application 
includes: 

computer readable program code for causing said client 
workstation to issue, from the client workstation, one or 
more commands to the network server; 

computer readable program code for causing said network 
server to execute one or more instructions in response 
to said commands; 

computer readable program code for causing said network 
sever to send information to said client workstation in 
response to said executed instructions; and 

computer readable program code for causing said client 
workstation to process said information at the client 
workstation to interactively control said controllable 
application. 

10. The computer program product of claim 9, wherein 
said additional instructions for controlling said controllable 
application reside on said client workstation. 
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